Improving Performances of BoW-based Image Retrieval by Using Contextual Keypoint Descriptors

The paper reports an improved method of content-based image retrieval using a well-known method of bag-of words (BoW). Words built over descriptors of popular affine-invariant keypoint detectors (Harris-Affine and Hessian-Affine are exemplary choices) are used. What is novel, however, is the number of descriptors (i.e. the number of words) representing individual keypoints. Instead of SIFT (or another alternative descriptors SIFT-like descriptors representing both visual properties of keypoints and their local configurations are proposed (adopted from our previous works). In average, each keypoint has 10-15 such descriptors, but the increased size of BoW representation is in our opinion acceptable because of significant performance improvements in BoW-based image retrieval, as shown in a feasibility study on a popular benchmark dataset. Such an improvement is possible because very large vocabularies can be built over the proposed descriptors without compromising the sensitivity of words to minor geometric and photometric distortions.